Goto

Collaborating Authors

 ci test



Federated Causal Discovery Across Heterogeneous Datasets under Latent Confounding

Hahn, Maximilian, Zajak, Alina, Heider, Dominik, Ribeiro, Adèle Helena

arXiv.org Machine Learning

Causal discovery across multiple datasets is often constrained by data privacy regulations and cross-site heterogeneity, limiting the use of conventional methods that require a single, centralized dataset. To address these challenges, we introduce fedCI, a federated conditional independence test that rigorously handles heterogeneous datasets with non-identical sets of variables, site-specific effects, and mixed variable types, including continuous, ordinal, binary, and categorical variables. At its core, fedCI uses a federated Iteratively Reweighted Least Squares (IRLS) procedure to estimate the parameters of generalized linear models underlying likelihood-ratio tests for conditional independence. Building on this, we develop fedCI-IOD, a federated extension of the Integration of Overlapping Datasets (IOD) algorithm, that replaces its meta-analysis strategy and enables, for the fist time, federated causal discovery under latent confounding across distributed and heterogeneous datasets. By aggregating evidence federatively, fedCI-IOD not only preserves privacy but also substantially enhances statistical power, achieving performance comparable to fully pooled analyses and mitigating artifacts from low local sample sizes. Our tools are publicly available as the fedCI Python package, a privacy-preserving R implementation of IOD, and a web application for the fedCI-IOD pipeline, providing versatile, user-friendly solutions for federated conditional independence testing and causal discovery.


Appendix A PCMCI Algorithm

Neural Information Processing Systems

The PCMCI algorithm is proposed by Runge et al. [2019], aiming to detect time-lagged causal See Fig.1 for more detail. A simple proof is shown below through Markov assumption ( A2). 3 Figure 2: Partial causal graph for 3-variate time series Fig.2 shows a partial causal graph for a 3-variate time series with Semi-Stationary SCM. However, they may not share the same marginal distribution. Still in Fig.2, based on the definition of homogenous time partition, time partition subset Based on Eq.(12) and Eq.(17), we have: p(X Without loss of generality, we assume T is a multiple of δ all the time. A1-A7 and with an oracle (infinite sample size limit), we have that: null G = G (47) almost surely.



A Supplement

Neural Information Processing Systems

Here we provide proofs of the statements made in the main text as well as further figures of numerical experiments and a more detailed discussion of heteroskedasticity effects regarding causal discovery. Z. Testing whether the Pearson correlation between X and Y is zero is equivalent to testing whether the slope parameter β is equal to zero. Therefore, this is a homoskedastic problem. A.1.2 Discussion of Effect 2: We start by discussing the homoskedastic case to see where non-constant variance of noise leads to problems within the t-test. For homoskedastic noise the second factor is an estimator of the standard error of ˆβ, which we derive by using the mean of the squared residual as an estimator for the error variance.






144a3f71a03ab7c4f46f9656608efdb2-Paper.pdf

Neural Information Processing Systems

Understanding the underlying mechanisms is crucial for tasks such asexplaining aphenomenon, predicting, anddecision making. Pearl(2009) providedamachinery for automating the process of answering interventional and (retrospective) counterfactual queries even when only observed data is available, and determining if a query cannot be answered given the available data type (identifiability). This requires knowledge about the true underlying causal structure; however,inmanyreal-world situations, thisstructure isunknown.